Regulation toward Self-organized Criticality in a Recurrent Spiking Neural Reservoir

The goal of the reservoir is to favor rich internal dynamics and interactions with the input signals, as to achieve the required fading memory and kernel properties [1]. There is however no general agreement yet on how the connectivity matrix of the reservoir should be initialized to yield both stability and performance. It is known that edge-of-chaos regime in recurrent neural network leads to maximal computation capabilities [2]. The main problem is that the transition between order and chaos is particularly sharp for spiking or binary neural networks [3], thus a difficult regime to achieve. Reservoir tuning may be performed globally in the case of echo state networks by normalizing the connectivity matrix by the spectral radius. However, the spectral radius does not influence much the performance in the case of liquid state machines [4]. Alternatively, a local and unsupervised tuning rule related to branching factor analysis has already shown success in stabilizing a recurrent spiking neural networks [5, 6]. An extended learning rule based on this form of self-organized criticality is now proposed. It is shown to tune a reservoir toward the edge-of-chaos in response to inputs specific to the task at hand. The aspect of input specificity is speculated to be very important, because the response of the reservoir is driven by both the input and the internal dynamics which reflect past inputs. In the high-dimensional projective space of the reservoir, only a subspace may actually be covered by the generated trajectories. Tuning the reservoir for particular inputs may thus increase sensibility to those trajectories, leading possibly to a higher memory capacity.